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An apparatus and method of generating an audio recording having linked data 



(57) A method is given for generating an audio recording 
including a linked second set of data that is input by the 
user during the audio recording. The second data set nf>ay 
by Itnlced to a subsection of the audio data. Flags may be 
placed to marlc certain points on the audio record so that 
when the audio record is replayed, the second data set is 
presented when the subsection is output The second data 
set is preferably input by a iceyboard, video data, a stylus, 
or networked data. The data sets are preferably temporally 
linked. In a preferred embodiment it is possible to output 
the audio data as visual data as textual data, while still 
allowing the second data set to be displayed at the 
appropriate time by placing flags. Also given is an 
apparatus 100 for carrying out the method, /^paratus 100 
has a stylus 104 to input handwritten text 106 on a touch 
sensitive display 102. Also provided is a microphone 108. 
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AN APPARATUS AjYD METHOD OF GENERATING 
AN A UDIO RECORDING INCLUDING UNKED DATA 

FIELD OF THK TNVF.NTTOM 
5 The present invention generally relates to the field of interactive media, and 

particularly to an apparatus and method of generating an audio recording iiwluding 
linked data. 

BACKGROUND OF THF. mVENTTON 

10 Summarizing data for later review is one of the most common tasks performed 

in the learning experience. From business to education, it is often necessary to 
summarize data so that the data may be later reviewed in a more expedient manner. 
For example, note taking is one of the most arduous tasks a student faces while taking 
courses. Students usually take notes during lectures in order to summarize the 

15 information for study at a later time. Typically, a student would take notes of the 
lecture to summarize relevant portions. However, one limitation with notes is that 
much of the content, and petiiaps accuracy, is lost in the summary. 

One method used to address this problem involves recording the lecture on 
audio data recorders. Some students may record the entire lecture on tape, in order to 

10 have the option of listening to part or even the entire lecture again..' However, such a 
recording did not contain other relevant data desired by the student, such as visual 
diagrams utilized by the mstructor, and the like. Thus, the user was limited to general 
listening of the lecture. Additionally, even if the user also took notes, those notes had 
to be kept separately and did not provide a manner in which to view the notes with 

15 the corresponding point of the lecture. Therefore, the user, often studying at a point 
much later in time than when the notes were taken, had difficulty in relating the notes 
with the recorded lecture. 

Thus, it would be desirable to provide a system and method of generating an 
audio recording including linked data. 



SUMMARY OF THE INVENTION 
Accordingly, the present invention is directed to an apparatus and method for 
providing an audio recording including linked data. In a first aspect of the present 
invention a method of generating an audio recordbg including linked data includes 
receiving a first set of audio data on an information appliance and receiving a second 
set of data input by a user on the information appliance. The second set of data input 
by the user is linked . to the first set of audio data received by the information 
appliance. Thus, upon output of the first set of audio data, the linked second set of 
data is output by the information appliance in conjunction with the output of the first 
set of audio data so as to enable access to both the first and second sets of data. 

In a second aspect of the present invention, a method for generating an audio 
recording including linked data includes receiving a set of audio data including a 
subsection of the set of audio data and receiving a second set of data. The second set 
of data is linked to tiie subsection of the first set of audio data. linking the data 
includes- placing a flag in die first set of audio data indicatiing the linked second set of 
data so that when die audio data is output, the second set of data is presented when 
the subsection of the first set of audio data is output 

In a third aspect of the present invention, an apparatus for generating an audio 
recording including linked data includes an audio recording device suitable for 
receiving a set of audio data including a subsection of the set of audio data. An input 
device suitable for receivmg a second set of data is coupled to the audio recording 
device, wherein the second set of data is linked to the section of the first set of audio 
data by placing a flag in the section of die first set of audio data indicating die linked 
second set of data. Thus, when the audio data is output, the second set of data is 
presented when the section of the first set of audio data is output. 

It is to be understood tiiat both the forgoing general description and the 
following detailed description are exemplary and explanatory only and are not 
restrictive of the invention as claimed. The accompanying drawings, which are 
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incorporated in and constitute a part of the specification, illustrate an embodiment of 
the invention and together with the general description, serve to explain the principles 
of the invention. 

5 BRIEF DESCRIPTION OF THE DRAWINGS 

The numerous advantages of the present invention may be better understood 
by those skilled in the art by reference to the accompanying figures in which: 

FIG. 1 is an isometric drawing illustrating an exemplary embodiment of the 
present invention wherein an information appliance is configured as a note taking 
10 apparatus suitable for receiving audio data; 

FIG. 2 is an isometric drawing further illustrating the exemplary embodiment 
as shown in FIG. 1, wherein the information appliance displays links suitable for 
accessing a second set of data from a first set of audio data; 

FIG. 3 is an isometric drawing further illustrating the exemplary embodiment 
15 of the present invention as shown in FIG. 1, wherein audio data received by an 
information appliance is converted to textual data suitable for display on the 
information appliance and suitable for linking to a second set of data; 

FIG. 4 is a flow diagram depicting an exemplary method of the present 
invention wherein a set of audio data includes a linked second set of data input by a 
20 user; 

FIG. 5 is a flow diagram illustrating an exemplary method of the present 
invention wherein a second set of data including user notes is linked to a first set of 
recorded audio data; 

FIG. 6 is a flow diagram illustrating an exemplary method wherein a user may 
25 specify an input device for receiving a second set of data suitable for linking to a first 
set of audio data; 

FIG. 7 is a flow diagram depicting an exemplary method wherein a second set 
of data is linked to a first set of audio data by placing a flag in the first set of audio 
data; 
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FIG. 8 is a flow diagram depicting an exemplary embodiment wherein a 
second set of data is linked to a first set of audio data utUizing a flag so as to enable 
the second set of data to be entered subsequently to the placement of the flag yet stUl 
enable the sets of data to be presented in a maimer corresponding to the placement of 
5 the flag; 

HG. 9 is a flow diagram illustrating an exemplary method wherein a set of 
audio data received by an information appUance is converted to a textual display 
suitable for linking by a second set of data; and 

FIG. 10 is a block diagram of an exemplary information appliance operable to 
10 employ the present invention. 

DETAILED D HSCRIPTTON QF THE TNVFNTTnM 
Reference wiU now be made in detail to the presently preferred embodiments 
of the invention, examples of which are iUustrated in the accompanying drawings. 
15 Refening generaUy now to RG. 1. an exemplary embodiment of the present 

invention is shown wherein an apparams suitable for generating an audio recording 
including linked data is shown. An apparatus 100 of the present invention may be 
configured as an information appUance suitable for mobile operation by a user. The 
apparatus may include a video display device 102 for displaying visual information. 
Additionally, it may be desirable to incorporate a touch sensitive surface on the video 
display 102 to enable a user to input data by interacting direcUy with the display 102. 
For example, a user may utUize a stylus 104 to input handwritten text 106 on the 
display 102. Preferable, the information appliance 100 employs optical character 
recognition software to convert the handwritten text to data understandable by the 
appliance, thus enabling the appliance to manipulate the data. In this way, a user may 

input handwritten notes during the course of a presentation, speech, lecture, and the 
like. 

Further, the information appUance 100 may include an audio input device, 
such as a microphone 108. so as to enable the apparatus 100 to record audio data. It 
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may also be preferable to include two audio input devices, wherein the first audio 
device 108 is optimized for recording audio data as given by a speaker in a lecture 
and the second audio input device 1 10 is optinuzed for receiving speech from the 
user. It may be preferable to include the second audio input device 110 as a part of 
5 the stylus 104, thus enabling the user to bring the audio input device 1 10 closer to the 
user to provide a better recording. 

Referring now to FIG. 2, an additional view of the exemplary embodiment of 
FIG. 1 is shown wherein the information appliance 100 displays links suitable for 
accessing a second set of data from a first set of audio data. A user may enter notes 

io on the information appliance related to a presentation. For example, the notes may 
include the well-known formula of energy equals mass multiplied by the speed of 
light squared 1 12 with the use of a stylus 104. The information appliance 100 may 
receive these notes and link them to the relevant points of the lecture. For instance, as 
the lecturer explains the theory, the notes 1 14 taken by the user may also be recorded. 

15 Upon playback of the lecture on the information appliance 100, the notes may be 
displayed in a temporal relationship corresponding to when the notes were written. In 
this way the user may interact with the recorded data in an improved maimer. 

There are a wide variety of types of data that may be linked to an audio 
recording without departing from the spirit and scope of the present invendon. For 

20 example, a user microphone may be provided for recording the user's comment with 
the lecture separately 110, Thus, the user may listen to the lecturer with his own 
comments, the lecturer alone, or the user notes alone, as desired. It may be desirable 
to display the linked sets of dau as representations 116 on the display device 102 to 
enable a user to access the data as desired. Further, the user may capture video data 

25 with the user of a video capture device and link that data 1 18 to the audio recording. 
For instance, the user may link a photo involving energy to mass 118 and/or a video 
recording of data displayed on a screen 120 by a lecturer and recorded by the user, 
and the like. It may also be preferable to utilize a "clip" button 122 to initiate the 
recording of the second set of data so as to provide just the portion of the data as 
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desired by the user. Moreover, network data niay also be linked to the audio 
recording as desired such as by providing an address 124 on the World Wide Web to 
related infonnation. 

A method for searching the linked data may also be desirable to provide an 
5 expedient and efficient manner of finding desired subsections of the data. For 
example, a search button 126 may be provided to initiate a searching utility. The 
search may be performed per type of data such as visual, audio, network, and the like, 
a temporal search wherein a user may choose from a plurality of linked data based on 
the corresponding time that first set of data was recorded and/or linked and the like as 
10 contemplated by a person of ordinary skill in the art. 

Refening now to FIG. 3, the exemplary apparatus 100 of FIG. 1 is shown 
wherein audio data received by an infonnation appUance is converted to textual data 
suitable for display on the infonnation appliance and is also suitable for being linked 
by a second set of data. The information appUance receives audio data. The audio 
15 data is then converted to textual data and displayed in a window on the display device 
of the information appUance. A user, wishing to Unk additional data to the audio 
data, may utilize the "dip" button 122 to flag a subsection 130 of the text 128 to 
which the additional data set is to be linked, and the Uke as contemplated by a person 
of ordinaiy skill in the art. 

Refening now to FIG. 4, an exemplary method 400 of the present invention is 
shown wherein a set of audio data includes a Unked second set of data input by a user. 
An information appUance receives a first set of audio data 402. Receiving audio data 
may include recording audio data on the information appliance, receiving audio data 
recorded by another recording device and transmitted to the information appliance, 
and the Uke as contemplated by a person of ordinary skUl in the art. A user of die 
information appUance 404 inputs a second set of data. Inputting the second set of 
data may include using a stylus and/or keyboard to enter data, receiving speech input 
spoken by a user, receiving visual data such as video and still pictures. Unking to 
network sites such as web sites on the Internet, and the Uke. The second set of data 
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input is linked to the first set of audio data received by the information appliance 406. 
By linking the sets of data, the linked second set of data is suitable for output in 
conjunction with the output of the first set of audio data 408. In this way, a user may 
interact with both sets of data. 
5 For example, as shown in FIG. 5, an exemplary method 500 includes linking a 

second set of data including user notes to a set of recorded audio data. A user may 
take an information appliance suitable for use as a note-taking device, such as the 
example apparatus shown in FIGS. 1 through 3, to a lecture. The user may then 
initiate an audio recording session 502 to record the lecture given by an instmctor. A 
10 first set of audio data is then received by the information appliance 504 of the lecture 
being recorded. As the lecture is being received, a user may input notes into the 
audio recording taken by die information appliance 506. The notes are received by 
the information appliance as a second set of data 508, The second set of data input by 
the user may then be temporally linked to the first set of audio data received by the 
15 information appliance 510. For instance, as the user receives audio data, the user may 
wish to make a note corresponding to the audio data just heard. The user may link the 
user inputted notes to the relevant subsection of the audio data so that as the audio 
data is played, the second set of data input by the user is also output 512. Thus, the 
user hears the first set of audio data and experiences the second set of data in a 
20 manner corresponding to when the second set of data was input and the first set of 
audio data was received 514. In this way, a user may view the second set of data with 
the relevant audio data in a concise and optimized manner. 

Referring now to FIG. 6. an exemplary method 600 of the present invention is 
shown wherein a user may specify an input device for receiving a second set of data 
25 for linking to a first set of audio data. A user may initiate an audio recording session 
to record audio data 602, such as to record a presentation given at a convention,^ 
symposium, and the like. The user may then specify a desired input device 604, such 
as a user microphone 606 to record audio data from the user, a user input screen 608 
such as a touch sensitive screen, a video input device 610 for recording a digital 
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image or group of images, network data 612 such as a link to a web site, and the like 
as contemplated by a person of ordinary skill in the art. The information appliance 
may then receive data input by the user 614 from the selected input device. The user 
data is capable of being linked to the audio data received by the information appliance 
616. Thus, the second set of user data is suitable for output in conjunction with the 
first set of audio data 618. In this way, a user is able to utilize a variety of input 
devices for inputting various forms of data suitable for linking with a set of audio 
data. 

Referring now to FIG. 7, an exemplary embodiment 700 of the present 
invention is shown wherein a second set of data is linked to a first set of audio data by 
placing a flag in the first set of audio data. A set of audio data is recorded, the set of 
data including a subsection 702. A second set of data is also received 704. The 
second set of data is linked to the subsection of the first set of audio data by placing a 
flag in the first set of audio data indicating the linked second set of data 706. Thus, 
during the output of the first set of audio data, when the flagged subsection of the first 
set of audio data is output, the second set of data is presented 708. 

For example, as shown in FIG. 8, in an exemplary embodiment 800 of the 
present invention, a second set of data is linked to a first set of audio data utilizing a 
flag so as to enable the second set of data to be entered subsequendy to the placement 
of the flag yet still enable the sets of data to be presented in a manner corresponding 
to the placement of the flag. A user may bring an apparatus of the present invention 
to a lecture 802 for taking notes. A recording session is initiated 804 to record a set 
of audio data including a subsection 806. As the user listens to the lecture, a point 
may be made by the lecturer with which the user wishes to associate a note. To 
accomplish this, a user may place a flag in the notes 808 to indicate a second set of 
data that the user wishes to associate with the desired subsection of the first set of 
audio data. The user may then enter a note corresponding to the flag 810 and have 
that second set of data received by the apparatus 8 12. Therefore, when the first set of 
audio data is played, upon presentation of the flagged subsection of the first set of 
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audio data, the second set of data may also be presented 814. This enables a user to 
enter data subsequently to the receipt of the subsection of the first set of audio data 
yet still enable both sets of data to be output concurrently. 

Referring now to FIG. 9, an exemplary embodiment 900 of the present 
5 invention is shown wherein a set of audio data received by an infonnation appliance 
is converted to a textual display suitable for linking by a second set of data. A set of 
audio data including a subsection is recorded 902 by an infonnation appliance. The 
audio data is then converted to textual data suitable for display on the infonnation 
appliance 904, For instance, the textual data may be displayed in a window on a 

10 display device of the infonnation appliance as shown in FIG. 3. If the user wishes to 
link a note to the audio data displayed as textual data, the user may place a flag in the 
notes to link subsequently entered data to the desired subsection 906. For instance, a 
user may double tap a stylus proximally to a word, sentence, letter and the like of the 
displayed audio data to insert a flag, press a start clip button as shown in FIGS. 1 and 

15 2, and the Uke as contemplated by a person of ordinary skill in the arL By inserting a 
flag, the user is able to enter a note corresponding to the flag 908 yet still have the 
flag associated with the desired subsection. Therefore, the infonnation appliance may 
receive the second set of data input by the user 910 subsequently to the receipt of the 
first set of audio data yet still link the data sets. In this way, a first set of audio data 

20 miay be displayed, and when the flagged subsection of the first set of audio data is 
output, a representation may be displayed indicating the second set of data 912. 
Thus, the user may then access the second set of data from the presentation of the first 
set of data, thereby providing a comprehensive note taking and viewing experience. 

Referring now to FIG. 10, a hardware system operable to employ the present 

25 invention is shown. The hardware system shown in FIG. 10 is generally 
representative of the hardware architecture of an information appliance of the present 
invention. An information appliance may include an information handling system, 
convergence system, Internet appliance, and the like without departing from the spirit 
and scope of the present invention. A controller, for example, a processing system 
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1002. controls the information appliance 1000. The processing system 1002 includes 
a central processing unit such as a microprocessor or microcontroller for executing 
programs, performing data manipulations and controlling die tasks of die informadon 
appliance 1000. Communicadon widi die processing system 1002 may be 
implemented dirough a system bus 1010 for transferring informadon among die 
components of die informadon appUance 1000. The system bus 1010 may include a 
data channel for facilitadng informadon Qansfer between storage and odier peripheral 
components of die informadon appliance 1000. The system bus 1010 further 
provides die set of signals required for communication witii processing system 1002 
including a data bus, address bus, and control bus. The system bus 1010 may 
comprise any state of die art bus architecture according to promulgated standards, for 
example industry standard architecture (ISA), extended industry standard architecture 
(EISA), Micro Channel Architecture (MCA), peripheral component interconnect 
(PCI) local bus. standards promulgated by the Institute of Electrical and Electronics 
Engineers (IEEE) including IEEE 488 general-purpose interface bus (GPIB), TFFF. 
696/S-600, and so on. Furdiermore, die system bus 1010 may be compliant with any 
promulgated industry standard. For example, die system bus 1010 may be designed 
in conapliance with any of die following bus architectures: Industry Standard 
Architecture (ISA), Extended Industry Standard Architecture (EISA), Micro Channel 
Architecture, Peripheral Component Interconnect (PCI), Universal Serial Bus (USB), 
Access bus. IEEE P6394, Apple Desktop Bus (ADB), Concentration Highway 
Interface (CHI), Fire Wire, Geo Port, or Small Computer Systems Interface (SCSI), 
for example. 

Additionally, die information appliance 1000 includes a memory 1004. In one 
embodiment, memory 1004 is provided on SIMMs (Single In-line Memory Modules), 
while in anodier embodiment, memory 1004 is provided on DIMMs (Dual In-line 
Memory Modules), each of which plugs into suitable sockets provided pn a 
modierboard holding many of die odier components shown in FIG. 10. Memory 
1004 includes standard DRAM (Dynamic Random-.Access Memory), EDO (Extended 
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Data Out) DRAM. SDRAM (Synchronous DRAM), or other suitable memory 
technology. Memory 1004 may also include auxiliary memory to provide storage of 
instructions and data that are loaded into the memory 1004 before execution. 
Auxiliary memory may include semiconductor based memory such as read-only 
5 memory (ROM), programmable read-oiJy memory (PROM) erasable programmable 
read-only memory (EPROM), electrically erasable read-only memory (EEPROM), or 
flash memory (block oriented memory similar to EEPR014). 

The information appliance 1000 further includes a network connection device 
1006. The network connection device 1006 communicates between the information 

10 appliance 1000 and a remote device, such as external devices, networks, information 
sources, or host systems that administer a plurality of information appliances. For 
example, host systems such as a server or infomaation appliance, may run software 
controlling the information appliance, serve as storage for an information appliance, 
or coordinate software running separately on each information appliance. The 

15 network connection device 1006 may provide or receive analog, digital, or radio 
frequency data. The network connection device 1006 preferably implements industry 
promulgated architecture standards, including Recommended Standard 232 (RS-232) 
promulgated by the Electrical Industries Association, Infrared Data Association 
(IrDA) standards, Ethernet IEEE 802 standards (e.g., IEEE 802.3 for broadband and 

20 baseband networks, IEEE 802.3z for Gigabit Ethernet, IEEE 802.4 for token passing 
bus networks, IEEE 802.5 for token ring networks, IEEE 802.6 for metropolitan area 
networks, 802.66 for wireless networks, and so on). Fibre Channel, digital subscriber 
line (DSL), asymmetric digital subscriber line (ASDL), frame relay, asynchronous 
transfer mode (ATM), integrated digital services network (ISDN), personal 

25 communications services (PCS), transmission control protocol/Intemet protocol 
(TCP/P), serial line Internet protocol/point to point protocol (SLIP/PPP), Universal 
Serial Bus (USB), and so on. For example, the network connection device 1006 may 
comprise a network adapter, a serial port, parallel port, printer adapter, modem, 
universal asynchronous receiver-transniitter (UART) port, and the like, or use various 



wireless technologies or links such as an infrared port, radio-frequency (RF) 
communications adapter, infrared transducers, or RF modem. 

The information appliance 1000 further includes a display system 1012 for 
comiecting to a display device 1014. The display system 1012 may comprise a video 
display adapter having all of the components for driving the display device, including 
video random access memory (VRAM), buffer, and graphics engine as desired. The 
display device 1014 may comprise a liquid-crystal display (LCD), cathode ray tube 
(CRT), or may comprise alternative type of display technologies, such as a light- 
emitting diode (LED) display, gas or plasma display, or employ flat-screen 
technology. 

An information appliance 1(X)0 may further include an input/output (I/O) 
system 1016 for connecting to one or more I/O devices 1018. 1020, and up to N 
number of I/O devices 1022. Input/output system 1016 may comprise one or more 
controllers or adapters for providing interface functions between one or more of I/O 
devices 1018-1022. For example, input/output system 1016 may comprise a serial 
port, parallel port, network adapter, printer adapter, radio-frequency (RF) 
communications adapter, universal asynchronous receiver-transmitter (UART) port, 
and the like, for interfacing between corresponding I/O devices such as a mouse, 
joystick, trackball, trackpad, trackstick, infrared transducers, printer, modem, RF 
modem, bar code reader, charge-coupled device (CCD) reader, scanner, compact disc 
(CD), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), 
video capture device, touch screen, stylus, electroacoustic transducer, microphone, 
speaker, and the like. It should be appreciated that modification or reconfiguration of 
the information appliance 1000 of FIG. 10 by one having ordinary skill in the art 
would not depart from the scope or the spirit of the present invention. For example, 
input/output system 1016 may include devices typically utilized only for input, such 
as a microphone, video capture device, and the like. 

Although the invention has been described with a certain degree of 
particularity, it should be recognized that elements thereof may be altered by pereons 
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skilled in the art without departing from the spirit and scope of the invention. One of 
the embodiments of the invention can be implemented as sets of instructions resident 
in the memory 1004 of one or more information appliances configured generally as 
described in FIG. 10. Until required by the information appliance, the set of 
5 instructions may be stored in another readable memory device, for example in a hard 
disk drive or in a removable memory such as an opdcal disc for utilization in a CD- 

ft 

ROM drive, a floppy disk for utilization in a floppy disk drive, a floppy/optical disc 
for utilization in a floppy/opdcal drive, or a personal computer memory card for 
utilization in a personal- computer card slot. Further, the set of instructions can be 

10 stored in the memory of an information appliance and transmitted over a local area 
network or a wide area network, such as the Internet, when desired by the user. 
Additionally, the instructions may be transmitted over a network in the form of an 
applet that is interpreted or compiled after transmission to the computer system rather 
than prior to transmission. One skilled in the art would appreciate that the physical 

15 storage of the sets of instructions or applets physically changes the medium upon 
which it is stored electrically, magnetically, chemically, physically, optically or 
holographically so that the medium carries computer readable information. 

In exemplary embodiments, the methods disclosed may be implemented as 
sets of instructions or software readable by a device. Further, it is und^tood that the 

20 specific order or hierarchy of steps in the methods disclosed are examples of 
exemplary approaches. Based upon design preferences, it is understood that the 
specific order or hierarchy of steps in the method can be rearranged while remaining 
within the scope of the present invention. The accompanying method claims present 
elements of the various steps in a sample order, and are not meant to be limited to the 

25 specific order or hierarchy presented. 

It is beUeved that the apparams and method of generating an audio recording 
including linked data of the present invention and many of its attendant advantages 
will be understood by the forgoing description. It is also believed tiiat it will be 
apparent that various changes may be made in the form, construction and 
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arrangement of the components thereof without departing from the scope and spirit of 
the invention or without sacrificing all of its material advantages. The form herein 
before described being merely an explanatory embodiment thereof. It is the intention 
of the following claims to encompass and include such changes. 
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CLAIMS 

What is claimed is: 

1 1 . A method of generating an audio recording including linked data, comprising: 

2 receiving a first set of audio data on an information appliance; 

. 3 receiving a second set of data input by a user on the information appliance; 

4 and 

5 linking the second set of data input by the user to the first set of audio data 

6 received by the information appliance, wherein upon output of the first 

7 set of audio data, the linked second set of data is output by the 

8 information appliance in conjunction with the output of the first set of 

9 audio data so as to enable access to both the first and second sets of 
10 data. 

1 2. The method as described in claim 1, wherein the second set of data includes at 

2 least one of data input by a user utilizing a stylus, data input by a user utilizing 

3 a keyboard, video data and a link to data accessible over a network. 

1 3. The method as described in claim 1, wherein the link includes temporally 

2 linking the second set of data as input by the user with the jSrst set of audio 

3 data as it is received by the information appliance. 

1 4. The method as described in claim 3, wherein the first set of audio data is 

2 output by the information appliance, the second set of data input by the user is 

3 output in a manner corresponding temporally to when the first set of audio 

4 data and the second set of data input by the user were received. 



1 

2 



The method as described in claim 1, further comprising converting the first set 
of audio data to textual data so as to enable the first set of audio data to be 



output in a visual manner. 



The method as described in claim 5, wherein the visual output of the first set 
of audio data is displayed concurrently with the second set of data input by the 
user, the second set of data including visual data, so as to enable a user to 
view both the visual output of the first set of audio data with a visual output of 
the second set of data. 

The method as described in claim 1, wherein linking includes placing a flag in 
a section of the first set of audio data indicating the linked second set of data 
so that when the audio data is output, the second set of data is presented when 
the section of the first set of audio data is output. 

The method as described, in claim 7, wherein the flag is placed in the set of 
audio data before the second set of data is received. 

The method as described in claim 8, wherein the placement of the flag in the 
set of audio data before the second set of data is received enables a user to 
input the second set of data subsequently to the recording of the subsection of 
the set of audio data, yet upon output of the subsection of the set of audio data, 
the second set of data is presented corresponding with the subsection of the 
first set of audio data. • 
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1 10. A method for generating an audio recording including linked data, 

2 comprising: 



3 receiving a set of audio data including a subsection of the set of audio data; 

4 receiving a second set of data; and 

5 linking the second set of data to the subsection of the first set of audio data, 

6 wherein linking includes placing a flag in the first set of audio data 

7 indicating the linked second set of data so that when the audio data is 

8 output, the second set of data is presented when the subsection of the 

9 first set of audio data is output. 

1 11. The method as described in claim 10, wherein the flag is placed at least one of 

2 before the subsection of the set of audio data and in the subsection of the set 

3 of audio data. 

1 12. The method as described in claim 10, wherein the flag is placed in the set of 

2 audio data before the second set of data is received. 

1 13. The method as described in claim 12, wherein the placement of the flag in the 

2 set of audio data before the second set of data is received enables a user to 

3 input the second set of data subsequently to the recording of the subsection of 

4 the set of audio data, yet upon output of the subsection of the set of audio data, 

5 the second set of data is presented corresponding with the subsection of the 

6 first set of audio data. 

1 14. The method as described in claim 10, wherein the second set of data includes 

2 at least one of data input by a user utihzing a stylus, data input by a user 

3 utilizing a keyboard, video data and a link to data accessible over a network. 

1 15. The method as described in claim 10, wherein the link includes temporally 
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linking the second set of data as input by the user with the first set of audio 
data as it is received by the information appliance. 

The method as described in claim 15, wherein the first set of audio data is 
output by the information appliance, the second set of data input by the user is 
output in a manner corresponding temporally to when the first set of audio 
data and the second set of data input by the user were received. 

■ 

The method as described in claim 10, further comprising converting the first 
set of audio data to textual data so as to enable the furst set of audio data to be 
output in a visual maimer. 

The method as described in claim 17, wherein the visual output of the first set 
of audio data is displayed concurrently with the second set of data input by the 
user, the second set of data including visual data, so as to enable a user to 
view both the visual output of the first set of audio data with a visual output of 
the second set of data. 
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1 19. An apparatus for generating an audio recording including linked data, 

2 comprising: 

3 an audio recording device suitable for receiving a set of audio data including a 

4 subsection of the set of audio data; and 

5 an input device suitable for receiving a second set of data is coupled to the 

6 audio recording device, wherein the second set of data is linked to the 

7 section of the first set of audio data by placing a flag in the section of 

8 the first set of audio data indicating the linked second set of data so 

9 that when the first set of audio data is output, the second set of data is 
10 presented when the subsection of the first set of audio data is output. 

1 20. The apparatus as described in claim 19, wherein the flag is placed at least one 

2 of before the subsection of the set of audio data and in the subsection of the 

3 set of audio data. 

1 21. The apparatus as described in claim 19, wherein the flag is placed in the set of 

2 audio data before the second set of data is received. 

1 22. The apparatus as described in claim 21, wherein the placement of the flag in 

2 the set of audio data before the second set of data is received enables a user to 

3 input the second set of data subsequently to the recording of the subsection of 

4 the set of audio data, yet upon output of the subsection of the set of audio data, 

5 the second set of data is presented corresponding with the subsection of the 

6 first set of audio data. 

1 23. The apparatus as described in claim 19, wherein the second set of data 

2 includes at least one of data input by a user utilizing a stylus, data input by a 

3 user utilizing a keyboard, video data and a link to data accessible over a 

4 network. 
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1 24. 
2 



The apparatus as described in claim 19, wherein the link includes temporally 
linking the second set of data as input by the user with the first set of audio 
3 data as it is received by the information appliance. 

1 25. The apparatus as described in claim 24, wherein the first set of audio data is 

2 output by the information appUance, the second set of data input by the user is 
3 

4 



output in a manner corresponding temporally to when the first set of audio 
data and the second set of data input by the user were received. 



1 26. The apparatus as described in claim 1 9. further comprising converting the first 

2 set of audio data to textual data so as to enable the first set of audio data to be 

3 output in a visual manner. 

1 27. The apparatus as described in claim 26, wherein the visual output of the first 

2 set of audio data is displayed concunendy with the second set of data input by 

3 the user, the second set of data including visual data, so as to enable a user to 

4 view both the visual ou^ut of the first set of audio data with a visual output of 

5 the second set of fiata 
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